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Abstract 

This study examined the effectiveness of the Executive 
Master of Business Administration (EMBA) degree 
program in terms of transfer of knowledge and leadership 
practices. Based on a review of literature related to adult 
learning theories, EMBA programs, the importance of 
evaluation practices, and leadership practices, this study 
was designed to determine whether an EMBA education 
can lead to the transfer of knowledge at workplaces to 
demonstrate and facilitate an employee’s leadership skills. 
Findings from this study showed that the curriculum and 
the transfer of knowledge were the most significant 
predictors of the use of leadership practices. Age and 
gender were found influential in the development of 
leadership skills. A significant finding was that curriculum 
was the most salient factor in determining the 
effectiveness of the transfer of knowledge and in 
promoting the use of leadership practices. 

Keywords: leadership, evaluation, curriculum, adult 
education, EMBA education 
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OVERVIEW OF THE STUDY 


In the past few decades, as business corporations have evolved 
to meet the needs of globalization, technological development, and 
the rapid exchange of information, the demand for advanced 
management education has increased. To succeed in these 
complex environments, managers cannot rely solely on what they 
learned as undergraduates and on what has been required by their 
employers if they are to enhance their management skills as team 
leaders, coaches, and mentors. Life-long learning in business 
organizations is not limited to management skills but also includes 
developing leadership skills that focus on enhancing the ability to 
apply knowledge to innovation and real-world problem solving. 
To satisfy the needs of coiporate employers, business schools have 
developed degree or non-degree programs, as well as custom 
programs for current and future business leaders. 

Clark and Estes (2002) noted the difference in business fields 
between education and information, job aid, and training. The 
benefits of information, job aid, and training are that they support 
employees 4 learning of how-to knowledge; whereas the benefit of 
education is that it provides trainees with the knowledge to solve 
novel and unanticipated problems. From the perspective of 
business organizations. Master of Business Administration (MBA) 
or Executive Master of Business Administration (EMBA) 
programs are long-term business investments for ensuring that 
leaders develop a set of leadership skills and possess the strategic 
management foresight to manage their teams. 

MBA and EMBA programs provide their graduates with 
returns on investment by contributing to their fast-tracking careers. 
However, there is little research regarding the relationship 
between transfer of knowledge and leadership practices and the 
extent to which participants apply that knowledge to practice and 
achieve organizational goals. One of the most important 
objectives of EMBA programs is to prepare middle- or senior- 
level managers to be outstanding managers, leaders, and 
professionals. While large amounts of money are spent each year 
for executive education to develop leadership competency, the 
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outcome has not been satisfactorily proved (Boyatzis & 
Saatcioglu, 2008). 


BACKGROUND OF THE PROBLEM 

Organizations and individuals make significant financial and 
human resource investments into MBA/EMBA programs. Thi s 
high cost of training and education has caused Human Resource 
Development (HRD) and training practitioners to focus on 
measuring whether these programs help their employees to close 
gaps in their management and leadership knowledge (Foreman, 
2008). Today, the assessment of management training programs is 
not limited to overall satisfaction or knowledge level, but also 
includes the transfer of learning to behavior and leadership 
practices. 

Many specialists have found that the Kirkpatrick model, which 
describes four basic level models (reaction, learning, transfer, and 
results), has been useful in evaluating professional educational 
programs (Clark & Estes, 2002). Management education should 
provide participants with an expansive reservoir of knowledge in 
learning situations, as well as the tools to solve real-life problems 
that lead to an immediate application of knowledge at their 
workplace. However, the application of knowledge and the 
outcome of knowledge in the Kirkpatrick four-level model have 
been neglected due to the great cost and effort expended by 
educators to evaluate the attainment of goals that are linked to 
participants ‘ achievements in professional education (Kirkpatrick 
& Kirkpatrick, 2006). 

According to Barbara Carpenter, head of Thunderbird‘s 
EMBA program, customized EMBA programs have been 
launched and developed in response to students 4 changing interests 
and companies 4 needs (Hansen, 2008). Hannum and Martineau 
(2008) emphasized the necessity of participants 4 critical 
knowledge, awareness, and ability to apply learning toward 
behavioral changes after the completion of leadership 
development programs. Chang-Chien (2005) reported that the 
EMBA curriculum was found to be positively related to training 
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transfer at the workplace. Consequently, universities should 
examine whether their programs meet the needs of business 
corporations to prepare future innovative business leaders and 
whether the programs satisfy the motivation of students in terms of 
their immediate and long-term desire to become effective 
managers. 


PURPOSE OF THE STUDY 

The purpose of this study was to examine the effectiveness of 
EMBA degree programs in terms of learning transfer and outcome 
factors at a large private four-year university in an urban city. This 
study sought to measure the degree to which participants in their 
leadership programs were able to transfer classroom learning to 
actual workplace behaviors and increase leadership skills. It also 
evaluated whether the EMBA programs, whose goals were 
developing leadership capabilities, met their objective by 
promoting the application of practical knowledge and tools to real- 
world situations to satisfy employers 4 needs. 

Specifically, this study explored how current EMBA students 
and graduates responded to four levels of evaluation based on the 
Kirkpatrick models and how they demonstrate their leadership 
practices at their workplace based on the Leadership Practices 
Inventory (LPI; Kouzes & Posner, 1988). Employees 4 
expectations regarding additional benefits as a result of completing 
the EMBA program were measured via an open-ended question. 


THEORETICAL FRAMEWORK 

Adult education theory has focused on differentiating itself 
from children's education. Most theorists agree that adults learn 
from situational experiences in their informal or incidental 
learning circumstances; however, they note the limitation of 
informal or incidental learning in terms of the extent to which 
adult students are ready to learn or are motivated to learn. 
Andragogy theory, which is a well- known theory of adult 
learning, is based on several assumptions. Many andragogy 
advocates have emphasized self-directed learning, immediate 
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application of knowledge, and cooperative learning with 
colleagues and facilitators (Glickman, C. D., Gordon, S. P., & 
Ross-Gordon, J. M., 2007). There are several debates regarding 
adult learning theories; however, adult learning theorists are in 
agreement that adult learning requires intellectual or manual 
processes as a means for change, development, and growth for the 
acquisition of professional knowledge or skills. 

As EMBA programs have proliferated in recent years, 
effective curricula have been required to meet employers ‘ need for 
competent managers and innovative team leaders. Thus, business 
schools have developed improved courses that are more immedi¬ 
ately relevant to their students 4 careers. However, there is still 
controversy about the quality of managers in business fields 
(Smith, 2007), and some researchers claim that there are still 
problems in training students to become effective managers in 
some areas such as in leadership capabilities. 

Kirkpatrick and Kirkpatrick (2006) have noted the importance 
of the evaluation process in checking the effectiveness of a 
program. Their evaluation process emphasizes how the program 
has contributed to an organization 4 s objectives and goals, whether 
to continue or discontinue the program, and how to improve the 
program. Kirkpatrick's four- level model (reaction, knowledge, 
transfer, and results) is still the most frequently used evaluation 
method in leadership development programs since the system 
measures overall satisfaction, improvement of transfer of learning, 
and outcome in business sectors. Evaluation at the results level 
measures organizational growth, such as increased sales, 
productivity and profits, as well as personal growth, such as 
improved interpersonal communication skills/ human relations, the 
quality of work life, leadership, time management, and decision¬ 
making capabilities. 

American companies spend over 100 billion dollars each year 
on training and development activities (Fareman & Ban, 1993). 
However, there are still doubts as to whether the financial 
resources were well spent toward achieving positive changes in 
individual and organizational behavior. Surprisingly, there are 
few systematic evaluation practices to test the linkage between 
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training and transfer of learning to behavior in the workplace. 
Newstrom (1986) claimed that too much money and attention has 
been spent on the design and delivery of programs and that little 
time and money have been allocated to evaluation practices for 
measuring the transfer of learning to the work environment. 
Likewise, there still has been little attention paid to assessing 
whether participants are changed in their attitudes, behavior, and 
performance through management education. 

There have been arguments among academics and 
practitioners regarding evaluation practices. There is little 
empirical evidence linking the transfer of learning to changes in 
performance and there is limited guidance on how to conduct 
evaluations (Fareman & Ban, 1993). Over the past decades, 
public and private organizations have initiated the measurement of 
outcome factors to justify the effectiveness of the adult education 
or training programs; consequently, they have been interested in 
evaluating the impact of training programs. Evaluators in business 
corporations and private consulting companies have relied on the 
Kirkpatrick four-level evaluation to measure and improve 
organizational productivity and to enhance leadership capabilities. 

In EMBA programs, few studies have examined evaluation 
practices in regard to transfer of knowledge and leadership 
abilities due to the perceived complexity of level3 (transfer) and 
level 4(results) in Kirkpatrick‘s (1994) four-level model. There 
has been debate regarding Kirkpatrick‘s four-level evaluation in 
terms of feasibility and the fact that there is little empirical 
evidence to support the success of four-level evaluation in public 
or private sectors. However, Kirkpatrick‘s (1994) model is 
important in attempting to measure behavior change and outcomes 
for evaluating the success of the programs in adult education. 

Kouzes and Posner have created wealth of practical 
suggestions, commitments, exercises, and examples of leadership 
behavior in many fields (Middlehurst, 1989). Their interest in 
leadership practices focuses on leaders 4 behavioral changes which 
instill motivation in their followers. They have stressed that 
leadership can be learned and developed through trial and error, 
people, and education in the process of managing and leading. 
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They have emphasized soft skills such as managing people 
(Sumner et al., 2006) through values, motivation, and 
encouragement rather than hard systems such as rules, controls 
and procedures (Middlehurst, 1989). However, some researchers 
have pointed out that both hard and soft systems are prerequisites 
for organizational success. Kouzes and Posner (2007) have 
proposed five guidelines to would-be leaders who want to take on 
the challenge of implementing exemplary leadership practices: 
Model the Way, Inspire a Shared Vision , Challenge the Process, 
Enable Others to Act, and Encourage the Heart. These guidelines 
are invaluable in that the assessment process offers information to 
supervisors or managers about their strengths and weaknesses in 
terms of personality traits and ways to enhance their overall 
leadership capabilities. The Leadership Practices Inventory (LPl) 
developed by Kouzes and Posner (1988) has been widely used to 
measure leadership skills in business organizations. Despite some 
disputes regarding the five leadership practices, they are looked 
upon as a useful guide to academics and practitioners for thinking 
about leadership capabilities. 


METHODOLOGY 

This study explored how current students and graduates 
responded to the four levels of evaluation based on Kirkpatrick‘s 
(1994) model and how they demonstrated their leadership 
practices at their workplace based on Kouzes and Posner‘s (1988) 
framework. As an evaluation measurement was not created by 
Kirkpatrick, Chang-Chien‘s (2005) four levels (expectations, 
curriculum, work environment, and transfer of knowledge) as 
independent variables were applied to the conceptual framework 
of Kirkpatrick 4 s four levels (reaction, learning, transfer, and 
results), and used as surrogates for Kirkpatrick 4 s four levels. 
Kouzes and Posner's (1988) leadership practices were used as 
dependent variable (see Figure 1). 


18 



Expectations 



Figure 1. The basic research design. 

Sample and Population 

The sample consisted of 148 current EMBA students (43 
—Indents”) and recent graduates (105 —giduatcs”) of EMBA 
programs at a large private university in Southern California, 
where the curricula cover both traditional programs and 
application of knowledge to develop leadership capabilities. The 
mean age of the respondents was 40.75 years, and they averaged 
18.26 years of work experience. The majority of those of 
participants (86.0%) were male. 


Instrumentation 

Two instruments were administered to all participants: 
Kirkpatrick 4 s Evaluation Model modified by Chang-Chien (2005) 
and the LP1 (see Figure 2). In addition, a demographic ques¬ 
tionnaire was included to provide information regarding age, 
gender, and years of work experience. The two questionnaires 
were used as major instruments in this study after receiving 
permission from Chang-Chien and Kouzes and Posner. 

The survey instrument, based on Kirkpatrick 4 s (1994) 
evaluation model, was a questionnaire adapted and modified from 
an instrument developed by R. F. Chang (2002) that was later 
modified and translated into English for a doctoral dissertation 
study at the University of Idaho by Chang-Chien (2005). She 
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modified the instrument to evaluate the effectiveness of Taiwan‘s 
current EMBA programs. In her measurement expectations, 
curriculum, work environment, and training transfer were utilized 
to evaluate the effectiveness of the EMBA curricula and delivery. 

Although her measurement levels are not exactly equivalent 
with Kirkpatrick 4 s levels, Kirkpatrick 4 s (1994) four-level 
framework was used as a benchmark for evaluation in her study 
(Chang-Chien, 2005). Chang-Chien conducted a pilot survey on 
the Internet using the translated questionnaire. Cronbach‘s alphas 
for sections 1 (expectations), 2 (curriculum), 3 (work 
environment), and 4 (training transfer) were .85, .81, .92, and .91, 
respectively. As the reliability of the instrument ranged from .81 
to .96 in the pilot study, her measurement was used as the major 
instrument for this current study. 

Covariates (Control Variables) 

Leadership practices cannot be examined solely from 
Kirkpatrick‘s four levels of evaluation. For this current study, in 
addition to the variables of expectations, curriculum, work 
environment, transfer of knowledge, and leadership practices, 
demographic variables (age, gender, and work experience) were 
used as controls. This enables the determination of the 
contribution of the predictor variables above and beyond the 
contributions of age, gender, and prior experience. 


Data Analysis 

This study used descriptive and inferential statistical data 
analyses to measure the transfer of knowledge and leadership 
practices. All quantitative data were coded and prepared for 
computerized analysis via the Statistical Package for the Social 
Sciences (SPSS®), Student version 15.0. Zero-order Pearson 
product correlation, two-tailed independent samples t tests, 
analysis of variance (ANOVA) with Welch statistics, and 
hierarchical multiple regression analysis were conducted. 
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Figure 2. The detailed research design. 

RESULTS 

All leadership practices were significantly related to one 
another at the .01 level. Curriculum was significantly correlated 
with all leadership practices at the .01 level, while transfer of 
knowledge was not related to any leadership practices except to 
enable others to act, r= AS,p < .05. 

Research question 1 asked: Do current students and 
graduates of the EMBA program differ in Kirkpatrick’s (1994) 
four-level evaluation scales as measured by expectations, curricu¬ 
lum,, work environment, and transfer of knowledge and Kouzes 
and Posner’s (1988) five leadership practices (modeling the way, 
inspiring a shared vision, challenging the process, enabling others 
to act, and encouraging the heart)? 


21 





























It would be expected that two-tailed t tests of independent 
samples, presuming a normal distribution, would have 
approximately the same variance. The results of the t tests in the 
present study showed different group variances in some variables. 
Therefore, a one-way ANOVA with Welch‘s statistics was 
conducted because it is insensitive to equality of variances and 
similarity of sample sizes. The results of the ANOVA showed that 
expectations differed significantly between current students and 
graduates, p = .05, and that the between-group difference in 
transfer of knowledge was marginally significant, p = .07. 

Results related to research question 1 showed that most of the 
leadership practice subscales, except model the way, were 
significantly different between groups (current students and 
graduates), indicating that graduate students were more likely to 
demonstrate leadership capacities after completion of the EMBA 
program. This result indicates that EMBA programs may improve 
the strategic skills and performance management of their graduates 
and increase their insight into their personal development as 
leaders. This is consistent with findings reported by London and 
Mone (1999) that EMBA programs promote leadership 
development. 

Research question 2 asked: To what extent are the four levels 
of Kirkpatrick’s evaluation (1994) related to Kouzes and Posner’s 
(1988) five leadership practices? 

Results of these correlation statistics for current students 
indicated that there were no significant relationships between 
Kirkpatrick‘s evaluation levels and exemplary leadership. 
However, curriculum was significantly associated with work 
environment r = .55, p < .01 and transfer of knowledge r = .32, p 
< .05. 

These results for the graduates showed that exemplary 
leadership had significant correlations with all evaluation 
variables. The support for the hypothesis that evaluation scales 
and leadership practices are significantly correlated was found in 
the case of graduates. 

Research question 3 asked: Do the four levels of Kirkpatrick’s 
(1994) evaluation predict leadership practices? Hierarchical 
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multiple regression analysis was conducted to predict Kouzes and 
Posner 4 s exemplary leadership practices as an outcome variable 
based on the responses of the four variables (expectations, 
curriculum, work environment, and transfer of knowledge) and 
the covariate variables (age, gender, and work experience) as 
predictor variables. As there was a significant difference between 
current students and graduates in exemplary leadership as 
mentioned in research question 2, hierarchical multiple regression 
analyses were conducted separately for current students and 
graduates. There were no predictors found for current students; 
however, there were significant predictors found for graduates. 

Results of hierarchical multiple regression analysis for 
graduates showed that the R 2 value (.527) indicates that about 
52.7% of the variance in exemplary leadership was explained by 
the seven predictor variables. The [> values indicate the relative 
influence of the entered variables; that is, curriculum had the 
greatest influence on exemplary leadership {[> =.498, p < .001), 
followed by transfer of knowledge (/? = .275, p < .01), gender (ft = 
.203, p < .05), and age (J3 = -.341, p < .05). 


Supplemental Data 

There were 15 responses from current students and 33 
responses from graduates. The results were as follows: 15 
respondents expected to increase new knowledge and skills, 18 
expected to gain social networks, and 14 expected to achieve 
career advancement. 

When taking into account the limitation of this study, a 
potential challenge was sample size only if there were insufficient 
results. As the sampling frame for current students and graduates 
was limited to one EMBA program, a low response rate could 
reduce the statistical power. Thus, the ability to detect real 
differences between the two groups may have been affected by the 
small and unequal sample sizes. 

The demographic information included age, gender, and work 
experience; however, additional data such as educational 

23 



background, certification, job category, and years in current 
position were overlooked in measuring leadership capabilities. 
Measuring leadership is complicated. It is possible that additional 
demographic information would have given a more holistic view 
of leadership skills and the effect that various demographic factors 
have on these skills. 


IMPLICATIONS OF THE FINDINGS 

Several implications emerge from the results of this study. 
First, curriculum appears to play the most significant role in 
predicting the use of leadership practices, reflecting a positive 
correlation. According to Robinson and Wick (1992), researchers 
have estimated that 70% of all leadership development occurs 
through informal, on-the-job experiences, while formal education 
programs contribute less than 10% to leadership development. 
Therefore, the finding of this study is important in pointing out 
that the role of formal education should be addressed because 
well-organized curricula are more likely to address the needs of 
business organizations in developing leadership. As with earlier 
formal learning theory, this study supports the importance of 
formal education. 

Second, Ormrod (2006) noted the importance of delivery and 
practice in adult education. If the objective is application, teachers 
should focus instruction and assessment methods on activities 
involving real objects and hands-on measurements for discovery 
learning that requires fostering transfer, problem solving, 
creativity, and self-regulated learning. Curriculum was the most 
significant predictor in terms of the use of leadership in this study; 
this is important because it gives practitioners or administrators in 
business schools a strategy for promoting best practices for the 
transfer of knowledge in EMBA programs. 

Third, the responses to the open-ended question support the 
practical needs of adult learners. Respondents 4 expectations 
included new knowledge/skills, access to broader social networks, 
and career advancement after the completion of their EMBA 
programs. This implies that adult learners are more likely to need 

24 



immediate achievement in a myopic way. The findings may guide 
administrators to consider how they can balance the curriculum by 
applying pedagogical and andragogical approaches. 

Last, the outcome assessment study based on Kirkpatrick‘s 
evaluation levels focusing on the learning process was conducted 
in management education. The results of this study should 
encourage school administrators to conduct outcome assessment 
studies to determine what their students are learning and what 
blocks the school mission of preparing people to manage and lead 
(Boyatzis & Saatcioglu, 2008). 


CONCLUSION 

Ottoson (1995) stated that educators and learners should 
consider the process by acknowledging the value of education and 
understanding that application and daily work are related but not 
synonymous concepts or theories. Hoffman (1981) labeled 
application as a key factor in adult education. In management 
education, application is most often understood as putting theories, 
principles, concepts, and ideas into practice. It deals with what is 
feasible, adaptable, and workable rather than with theoretical 
concepts. 

Adult learners are required to have contextual application 
knowledge, which entails flexibility. Glaser (1990) referred to a 
worker who demonstrates effective application of knowledge as a 
skillful thinker. However, application does not guarantee out¬ 
comes (Ottoson, 1995) in terms of how learners transfer what they 
have learned to novel workplace tasks or other situations. 

This study examined the relationship between EMBA 
students 4 transfer of knowledge and leadership practices to the 
workplace. The study investigated the major characteristics of 
EMBA programs regarding expectations, curriculum, work 
environment, and transfer of knowledge—all factors that may 
impact leadership skills. 
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The findings of this study imply that universities are still an 
invaluable source of knowledge and learning to produce successful 
outcomes in adult education. In other words, the curriculum 
design has a positive effect on students 4 transfer of knowledge that 
is closely related to the use of leadership practices. The results for 
the open-ended question show that approaches that are less 
theoretical and more practical are more likely to be welcomed by 
EMBA students. However, management education programs must 
balance academic and practical approaches in order to offer 
specific step-by-step instructions using an algorithm and a 
heuristic way of learning. This would enable students to seek not 
only to improve their self-development for credentialed job 
opportunities but also to develop long-term visions to be 
outstanding leaders in a turbulent globalized society. 

Few studies have examined evaluation practices based on 
Kirkpatrick 4 s four-level model (reaction, learning, transfer, and 
results). By linking the level of expectations, curriculum, work 
environment, transfer, and leadership practices to covariate 
variables (age, gender, and work experience), this study provides 
an empirical investigation showing that developing leadership 
skills is not simple and that knowledge and skill should be in place 
to facilitate the process of developing leaders. The results of this 
study are a significant step toward predicting and explaining the 
leadership development process in business organizations through 
evaluating the effectiveness of executive education programs. 
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